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ABSTRACT 

This paper presents a plan of action for managing 
evaluation implementation and ensuring the ongoing utilization of 
evaluation results. It addresses strategies that are required to 
manage the problems encountered in a 10-step management plan. Focus 
is on introducing an evaluation monitoring system within 
organizations. Steps in the plan include; (1) pre-evaluation 
assessment; (2) implementation of human resources strategies; (3) 
establishment of Key result areas, indicators, and performance 
measures, steuidards and targets, and other criteria for judging the 
program; determination of priorities for collecting research; and 
setting of boundaries on what the evaluation will produce; (4) 
development of a database management system; (5) determination of 
responsibility for evaluation and data collection emd manipulation; 
(6) estaOslishment of the report.lng and feedback requirements; (7) 
determination of rewards for performance; (8) initiation of 
evaluation procedures; (9) determination of the worth of the 
evaluation; and (10) maintenance of impetus via continued endorsement 
from management and widespread reporting of achievements. Rationale 
for such a plan cf miction, organizational structures that support 
evaluation, and designt:ition of responsible per^^'^ns are also 
discussed. (TJH) 
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ABSTRACT 



Many organizations expect that evaluation techniques will 
assist them to know what is happening in the operation of 
their organization, and as a result act as a watchdog to 
monitor the new "r^und autonomy of managers, cut costs, and 
provide a guide to the planning and delivery of more 
efficient and effective services. Although evaluation 
offers the opportunities to achieve the above, these 
successful outcomes depend upon careful and expert 
evaluation plannina and management. 

This paper presents a plan of action for managing evaluation 
implementation and ensuring the ongoing utilization of 
evaluation results. It addresses the strategies required to 
manage the problems encountered prior to and during the 
implementation process in a ten step management plan. 
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1 .0 INTRODUCTION 

Many organizations expect that evaluation techniques 
will assist them to know what is happening in the 
operation of their organization and as a result, act 
as a watchdog to monitor the new found autonomy of 
managers, cut costs and provide a guide to the 
planning and delivery of more efficient and effective 
services. 

Although evaluation offers the opportunity to achieve 
the above , these successful outcomes depend upon 
careful and expert evaluation planning and management. 

This paper presents a plan of action for managing 
evaluation i mpl ementati on and ensuring the ongoing 
utilization of evaluation results. It considers why 
we need a plan of action, what organization structures 
support the i n trod uct ion of eval uation, some 
al ternati ve forms for eval u at i on responsi bi 1 ity , and 
ten steps in an evaluation management plan. 



2.0 WHY DO WE NEED A PLAN OF A CTION FOR MANAGING 
EVALUATION ? 

There are at 1 east three reasons for pi anni ng an 
evaluation: 

1. A plan establishes the: 

- tasks to be accomplished 

- responsibility for their accomplishment 
timeframe 

- budget 

Without a plan our evaluation approach will be 
haphazard, lack co-ordination and fail to gain 
commi tment from managers and staff. The absence of 
guidel i nes can generate unreal i s tic expectations 1 n 
users which may be unmet because they are either too 
high or too low. 

2. A plan is a target against which to judge the 
success of the system. 

In contrast to uncertainty: 

- realistic expectations can be motivating 

- a plan anticipates and takes the steam out of 
problems before they start 
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3. The third purpose of a plan is to provide a 
framework : 

- to build evaluation into organizational 
structures and operational procedures 

to make evaluation a management tool in whi ch 
evaluation and monitoring are integrated into a 
continuous process of policy f ormati ons , goal 
setting, planning, budget ing, service deli very 
and evaluation. 



3.0 ORGANIZATION STRUCTURES THAT SUPPORT EVALUATION 

Evaluation is a management tool. Like any management 
practice, it wil 1 not survive unless supported by 
adequate organization structures. 

They include: 

1. An appropriate organization culture, i.e. one in 
which val ues , rol es and rul es endorse excel 1 ence 
of performance , performance measurement and 
rewards commensurate with performance. 

2. Open communication systems. In Kanter's (1985) 
terms they should be integrative rather than 
segmented. 

3. Provision of rewarding experiences for good 
performance. 

4. Management systems, including: 

* Stated goals, objectives 

* Data base management systems with standardized 
data col 1 ecti on, coding, storage, analysis, ?ind 
presentation 

* Shared evaluation phi^'osophy and practices: 

common understanding of what is meant by 
evaluation, and 

- agreement on the purpose of the evaluation 

- agreement on where evaluation fits into the 
planning process 
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4.0 



WHO_SHOULD BE RESPQNSIRl P Pn q EVALUATinN 7 

depend upon some Ders^n n'* "^^^^ P^Qanization will 

for its i.pllSenta't'Jon" °' ^"''^ ^'^^"^ responsibility 

Some options for this entity are: 

* Program Managers 

* Internal/External Consultants 



* Co-ordi nators 

* All of these 



Occasion depends'i;,r''tr'„''^' """^ Particular 

Each Of thffbove have dif?SrSJ." °f """"ion. 
different evalSHlor purposes 

S«e^of the tasks undertaker by these are presented 



Program Manager 



Internal Consultant- 



Sets goals, objectives 

Does formative evaluation 
and monitoring 

Acts tc impact on operations 

Designs data systems 

Selects monitoring data 

Codes, analyses, presents the 
aata in an appropriate for:n 

Provides knowledgeable 
feedback 
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Co-ordi nator or 

Steering Committee - Co-ordinates evaluation 

activities across programs 

Maintains some standardization 

Acts as an information channel 
between departments, and up 
and down programs and 
departments 

Is responsible for an annual 
evaluation report 

Provides evaluation skills and 
resources to p igram managers 

Trains managers and staff in 
eval uation 



External Consultant - Identifies objectives 

Provides credibility 

Provides assistance with the 
evaluation planning and 
implementation by recommending 
appropriate evaluation 
methodol ogy 

Provides training in 
evaluation skills 



Provides rei nforcem-nt 

Assists with setting 
objectives, di'fining 
performance measures 



Designs data coMection too^s 
^LJ/ llly.}^^K t"^er"aj consultants cannot manage thsse 

'tn ^^'^ ^° have sufficient 

time to meet all the demands of setting up a new and 
innovative structure. y a new ana 



There is usually a need for two types of evaluation; 
* Evaluation Monitoring used for decision making. 

- an integrated system 

" S.JilfJ-'" tj^^ serves policy planning and 
budgeting decisions a " 

" ^iJVi^l^''^ system, i.e. indicators are 
limned to a useful, manageable number that 
« collected and available on 

JiSlw '"^ '^^^ ^° ^« presented 

wiaeiy and in an appropriate form 

^nnSnJlJ^^^^'^" Studies or Reviews that will be 
conducted in response to periodic needs for 
evaluation or in response to particular needs 

?esSonV'i'.ni?! evaluations may use similar data but 
wi?h Jhf ^ ^^"^ measuring a program often rests 
with the program manager who has control of I 
program's operations. In contrast, the puroose of 
?; e';S;il7'V7edJbl.°'''" prese'nt'an 'ob7ective 

of^7oriVi^call7^7n7rtive°^ssle..^^°^-^ 



IS%vaVurti^on".^°^. P'P"" ^"^ introduction 

an evaluation monitoring system. 
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5-0 WHAT ARE THE STEPS IN MANAGING EVALUATION? 




ten steps in planning and managing 

* 



STEP 1 



PRE-EVALUATION ASSESSMENT 



The tasks are: 

Understanding the context of the program 
- determining what is the program - its 
logic, structure, inputs, processes 
(activities, decisions, throughputs) 
that lead to results - outputs, outcomes 
and impacts 

Conc'ucting an organizational climate 
survey to generate or identify 
readiness, awareness, constraints and 
support 

Determining the resources available for 
eval udti on 

Understanding the needs of the users of 
the evaluation monitoring system: 

' are the users? 

What do they want to evaluate? 

Do they want to make de'isions 
about program efficiency, 
effectiveness, appropriateness, 
standards of service provision, 
or a!l of these? 

Do they want to address specific 
i ssues? 

What level of the organization will 
the monitoring system serve? 

Designing the evaluation: assessing the 
appropriate evaluation method, data 
collection analysis and presentation 

Producing evaluation assessment report: 
recommending the evaluation brief or 
terms of reference, work plan. 



composition of evaluation 
identifying likely benefits. 



team. 
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STEP 2 IMPLEMENTING THE HUMAN RESOURCES STRATEGIES 
that will assist in building the appropriate 
supportive culture. These are described in 
a previous paper (Armstrong 1985). Briefly 
they are: 

(1) Endorsement of the evaluation by senior 
members of the organization e.g. by: 

a policy statement 
commitment and involvement 



(2) Compulsory legislative enforcement 

(3) Consultation - with key informants about 
the purpose of the evaluation, how they 
can be involved, how they want to be 
i n vol ved 

(4) Communication of Information about the 
evaluation: 

who is responsible 

who is conducting the evaluation 

who will collect, analyze, 
interpret data 

why evaluate 

what/who is being evaluated 

detailed plans and actions 

is it a review or a long term 
process 

what do managers have to do 

what are the benefits (what's in it 
for me?) 

Benefits accrue to the 
organization, individual members of 
the organization and clients. 



10 



Tne benefits to the organization vary 
from improved program performance 
relevant to "managi ng with less" to 
specific impacts on program needs. 

The benefits for each individual are: 

A sense of security (managers know 
what is expected and the extent to 
which they are achieving it) 

Improved self-esteem 

Improved QWL (increased 

responsibility and control of their 
work) 

A sense of worth from being aware 
of their contribution to the 
overall goals of their organization 

Benefits for "cl ients" are improved 
service delivery. 

The method of communication of the 
information is also important. 

Direct person to person contacts from a 
credible source is the best way of 
generating interest with key people. 
The use of mass media ana peer pressures 
from key people wil 1 build commitment 
among the majority. 

Traini ng: 

The successful operati on of an 
eva'^uation program depends upon the 
knowledge, skills and ex pert i se of the 
managers and staff. 

Some of the consponents of a tra1 ning 
program may include: 

knowledge of evaluation philosophy, 
principles and practices 

setting objectives 

defining key result areas, 
indicators and performance measures 

and especi al ly how to use 
eval uation information in deci sion 
maki ng 

Any program shoul d be supported by 
reinforcement and follow up activities. 
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STEP 3 * ESTABLISH the key result areas. 

Indicators and performance measures 
standards and targets, and other 
criteria for judging the program 

* DETERMINE the priorities for the 
collection of a minimum set of measures 
that are suitable for routine, regular 
and timely reporting 

* SET boundaries on what the evaluation 
will produce 

STEP 4 DESIGN A DATA BASE MANAGEMENT SYSTEM 

(A) - Software 

Hardware 

(B) Design the routine data collection 
coding, analysis and presentation 

STEP 5 DETERMINE RESPONSIBILITY FOR: 

* The evaluation 

* Data collection, manipulation 



^'^^^ ^ lll^rV^^ REPORTINb, 
PROCEDURES: 



FEEDBACK 



o5?a„ 'V results, when, how 

often, how will they be distributed? 
How will evaluation fit into the overall 
management cycle? 

STEP 7 DETERMINE HOW PERFORMANCE WILL BE REWARDED: 

Will it be tied to promotions, monetary 
rewards, fringe benefits 

- Will it invite some other means of 
recognition 

STEP 8 START THE EVALUATION PROCEDURES 

STEP 9 DETERMINE THE WORTH OF THE EVALUATION 

Meta-Evaluation against the "Standards 
for Evaluation" or pre-determi ned 
cr i teri a 
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STEP 10 MAINTAIN IMPETUS BY: 

Continued endorsement from senior 
management 

Widespread reporting and distribution of 
achievements 



CONCLUSION 

Most professionals know something about data collection. 
This plan of action addresses many of the problems 
encountered prior to and during the implementation process. 
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